Photographing apparatus, motion estimating apparatus, image compensating method, motion estimating method, and computer-readable recording medium

ABSTRACT

A photographing apparatus, a motion estimating apparatus, an image compensating method, a motion estimating method, and a non-transitory computer-readable recording medium are provided. The photographing apparatus includes: an image sensing unit which continuously captures a plurality of images by using a rolling shutter method; and an image processor which compensates for a uniformly accelerated motion of the photographing apparatus by using the plurality of images.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority under 35 U.S.C. §119 from Korean PatentApplication No. 10-2011-0102151, filed on Oct. 7, 2011, in the KoreanIntellectual Property Office, the disclosure of which is incorporatedherein by reference in its entirety.

BACKGROUND

1. Field

Apparatuses consistent with exemplary embodiments relate to aphotographing apparatus, a motion estimating apparatus, an imagecompensating method, a motion estimating method, and a computer-readablerecording medium, and more particularly, to a photographing apparatuswhich can check a uniformly accelerated motion thereof throughcontinuously captured images, a motion estimating apparatus, an imagecompensating method, a motion estimating method, and a computer-readablerecording medium.

2. Description of the Related Art

An imaging device refers to a part which generates an image in a mobilephone camera or a digital still camera (DSC). Examples of the imagingdevice include a charge-coupled device (CCD) image sensor and acomplementary metal oxide semiconductor (CMOS) image sensor.

The CCD image sensor refers to a device in which metal-oxide-silicon(MOS) capacitors are very close to one another, and charge carriers arestored in and transferred to the MOS capacitors. The CMOS image sensorrefers to a device which adopts a switching method of forming and usingMOS transistors by the number of pixels by using CMOS technology forusing a control circuit and a signal processing circuit as peripheralcircuits in order to sequentially detect outputs.

The CCD image sensor attracts the widest attention as a conventionalimage sensor and is currently widely used in a digital camera, a cameraphone, or the like. However, as the importance of the camera phonestands out, it is important to reduce power consumption of the cameraphone. Therefore, interest in the CMOS image sensor has increased. TheCMOS image sensor is manufactured in a CMOS process for producinggeneral silicon semiconductors, and thus, is small and cheap, and powerconsumption thereof is low.

A method of reading an optical image of a subject formed in an imagingarea of an imaging device is classified as a global shutter method and arolling shutter method. The global shutter method refers to a method bywhich all pixels of the imaging area read the optical image at a sametime. The rolling shutter method refers to a method by which one orseveral pixels of the imaging area sequentially read the optical image.

The CMOS image sensor may apply both to the global shutter method and tothe rolling shutter method. If the CMOS image sensor applies the globalshutter method, all pixels read the optical image of the subject at asame time. Therefore, even if the subject moves, a captured image is nottransformed.

If the CMOS image sensor applies the rolling shutter method, one orseveral pixels sequentially read the optical image. Therefore, if thesubject is moving or a photographing apparatus is being moved, acaptured image may be transformed. Accordingly, if a subject that ismoving is captured, a photographing apparatus applying the rollingshutter method has difficulty capturing a normal image.

A technique for compensating for an image distortion of the rollingshutter method is classified into an optical image stabilizer (OIS)technique and a digital image stabilizer (DIS) technique.

The OIS technique refers to a method by which a motion of aphotographing apparatus is measured by using a gyro-sensor, and an imagesensor or a lens of the photographing apparatus is moved in an oppositedirection in order to compensate for the motion of the photographingapparatus. The OIS technique is very effective, but is costly. Also, ifthe motion of the photographing apparatus exceeds a displacement rangeprovided by the OIS technique, there is no compensation for the rollingshutter phenomenon.

The DIS technique refers to a method by which a motion between images isestimated by using a digital signal processing method, wherein adistortion of an image is compensated for according to the estimatedmotion.

However, the DIS technique has problems with a calculation amount andaccuracy. In detail, a large amount of calculation is required toestimate a motion between images in a camcorder or a camera which is toprocess a moving picture in real time. Also, a motion of a photographingapparatus is estimated on the assumption that the photographingapparatus moves at a constant velocity. Therefore, an accuracy problemexists.

SUMMARY

One or more exemplary embodiments provide a photographing apparatus, amotion estimating apparatus, an image compensating method, a motionestimating method, and a computer-readable recording medium, all ofwhich can check a uniformly accelerated motion through continuouslycaptured images.

Additional aspects and advantages of the present general inventiveconcept will be set forth in part in the description which follows and,in part, will be obvious from the description, or may be learned bypractice of the general inventive concept.

The foregoing and/or other features and utilities of the present generalinventive concept may be achieved by providing a photographingapparatus. The photographing apparatus may include an image sensing unitwhich continuously captures a plurality of images by using a rollingshutter method, and an image processor which compensates a uniformlyaccelerated motion of the photographing apparatus by using the pluralityof images.

The photographing apparatus may further include a motion estimator whichestimates the uniformly accelerated motion of the photographingapparatus by using the plurality of images, and the image processor maycompensate for the plurality of captured images based on the estimateduniformly accelerated motion.

The motion estimator may include a comparator which compares theplurality of images with one another to check displacements among theplurality of images, a velocity calculator which calculates a uniformacceleration function of a motion velocity of the photographingapparatus according to the checked displacements, and a positioncalculator which calculates a position function of a position change ofthe photographing apparatus by using the calculated uniform accelerationfunction.

The comparator may compare pixel lines of each of the plurality ofimages to check a position of a pixel line in another image based on apixel line of one of the plurality of images in order to check adisplacement of the pixel line.

The comparator may compare preset pixel groups of pixel lines of each ofthe plurality of images to check a position of a pixel group in anotherimage based on a pixel group of one of the plurality of captured imagesin order to check a displacement of the pixel group.

The comparator may check a displacement of each of a plurality of pixellines in the image.

The velocity calculator may calculate the uniform acceleration functionof the motion velocity of the photographing apparatus based on anaverage of the displacements of the plurality of pixel lines.

The uniform acceleration function may be a linear function of time as inthe Equation below:v _(n)(t)=v _(n,0) +a(t−n) n≦t<n+1  (1)wherein t denotes time, n denotes a number of an image, Vn(t) denotes avelocity of the photographing apparatus at the time t, v_(n,0) denotesan initial velocity of the photographing apparatus at an n^(th) image,and a denotes an acceleration constant.

The initial velocity v_(n,0) of the photographing apparatus may becalculated by using the Equation below:v _(n,0)=GMV(n−1)−⅙(GMV(n)−2GMV(n−1)+GMV(n−2))wherein v_(n,0) denotes the initial velocity of the photographingapparatus at the n^(th) image, and GMV denotes a displacement of aparticular image.

The position calculator may integrate the calculated uniformacceleration function to calculate the position function of the positionchange of the photographing apparatus, wherein the position function isa quadratic function of time.

The motion estimator may estimate a uniformly accelerated motion of theimage sensing unit in one axis direction and a uniformly acceleratedmotion of the image sensing unit in a direction perpendicular to the oneaxis direction.

The motion estimator may estimate at least one of a uniformlyaccelerated motion in a yaw direction based on a center of the imagesensing unit, a uniformly accelerated motion in a pitch direction basedon the center of the image sensing unit, and a uniformly acceleratedmotion in a roll direction based on the center of the image sensingunit.

The motion estimator may divide a captured image into a plurality ofareas and estimate uniformly accelerated motions of the plurality ofareas.

The motion estimator may estimate uniformly accelerated motions of theplurality of images.

The foregoing and/or other features and utilities of the present generalinventive concept may also be achieved by providing a motion estimatingapparatus. The motion estimating apparatus may include an input unitwhich receives a plurality of images which are captured by using arolling shutter method, and a calculator which calculates a positionfunction of a position change of a photographing apparatus, whichcaptures the plurality of images, by using the plurality of images.

The calculator may include a comparator which compares the plurality ofimages with one another to check displacements among the plurality ofimages, a velocity calculator which calculates a uniform accelerationfunction of a motion velocity of the photographing apparatus accordingto the checked displacements, and a position calculator which calculatesthe position function of the position change of the photographingapparatus by using the calculated uniform acceleration function.

The calculator may calculate a first position function of thephotographing apparatus in one axis direction and a second positionfunction of the photographing apparatus in a direction perpendicular tothe one axis direction.

The calculator may calculate at least one of a first rotation functionof the photographing apparatus in a yaw direction, a second rotationfunction of the photographing apparatus in a pitch direction, and athird rotation function of the photographing apparatus in a rolldirection.

The calculator may estimate uniformly accelerated motions of theplurality of images.

The foregoing and/or other features and utilities of the present generalinventive concept may also be achieved by providing a method ofcompensating an image in a photographing apparatus. The method mayinclude continuously capturing a plurality of images by using a rollingshutter method and compensating a uniformly accelerated motion of thephotographing apparatus by using the plurality of captured images.

The method may further include estimating the uniformly acceleratedmotion of the photographing apparatus by using the plurality of images,wherein the plurality of images are compensated based on the estimateduniformly accelerated motion.

The estimation of the uniformly accelerated motion of the photographingapparatus may include comparing the plurality of images with one anotherto check displacements among the plurality of images, calculating auniform acceleration function of a motion velocity of the photographingapparatus according to the checked displacements, and calculating aposition function of a position change of the photographing apparatus byusing the calculated uniform acceleration function.

Pixel lines of each of the plurality of images may be compared to checka position of a pixel line in another image based on a pixel line of oneof the plurality of images in order to check a displacement of the pixelline.

Preset pixel groups of pixel lines of each of the plurality of imagesmay be compared to check a position of a pixel group in another imagebased on a pixel group of one of the plurality of images in order tocheck a displacement of the pixel group.

The displacement of a plurality of pixel lines in the image may bechecked.

The uniform acceleration motion of the motion velocity of thephotographing apparatus may be calculated based on an average of thedisplacements of the plurality of pixel lines.

The uniform acceleration function may be a linear function of time as inthe Equation below:v _(n)(t)=v _(n,0) +a(t−n) n≦t<n+1where t denotes time, n denotes a number of an image, Vn(t) denotes avelocity of the photographing apparatus at the time t, v_(n,0) denotesan initial velocity of the photographing apparatus at an n^(th) image,and a denotes an acceleration constant.

The initial velocity v_(n,0) of the photographing apparatus may becalculated by using the Equation below:v _(n,0)=GMV(n−1)−⅙(GMV(n)−2GMV(n−1)+GMV(n−2))where v_(n,0) denotes the initial velocity of the photographingapparatus at the n^(th) image, and GMV denotes a displacement of aparticular image.

A first uniform acceleration function of the photographing apparatus inone axis direction and a second uniform acceleration function of thephotographing apparatus in a direction perpendicular to the one axisdirection may be calculated.

At least one of a uniform acceleration function in a yaw direction basedon a center of the image sensing unit, a uniform acceleration functionin a pitch direction based on the center of the image sensing unit, anda uniform acceleration function in a roll direction based on the centerof the image sensing unit may be calculated.

A captured image may be divided into a plurality of areas, and uniformacceleration functions of the plurality of areas may be calculated.

The calculated uniform acceleration function may be integrated tocalculate the position function of the position change of thephotographing apparatus, wherein the position function is a quadraticfunction of time.

Uniform accelerated motions of the plurality of images may be estimated.

The foregoing and/or other features and utilities of the present generalinventive concept may also be achieved by providing a motion estimatingmethod. The motion estimating method may include receiving a pluralityof images which are captured by using a rolling shutter method, andcalculating a position function of a position change of a photographingapparatus, which captures the plurality of images, by using theplurality of images.

The calculation of the position function may include comparing theplurality of images with one another to check displacements among theplurality of images, calculating a uniform acceleration function of amotion velocity of the photographing apparatus according to the checkeddisplacements, and calculating the position function of the positionchange of the photographing apparatus by using the calculated uniformacceleration function.

A first position function of the photographing apparatus in one axisdirection and a second position function of the photographing apparatusin a direction perpendicular to the one axis direction may becalculated.

At least one of a first rotation function of the photographing apparatusin a yaw direction, a second rotation function of the photographingapparatus in a pitch direction, and a third rotation function of thephotographing apparatus in a roll direction may be calculated.

Uniformly accelerated motions of the plurality of images may beestimated.

The foregoing and/or other features and utilities of the present generalinventive concept may also be achieved by providing a computer-readablerecording medium comprising a program for executing a motion estimatingmethod. The motion estimating method may include receiving a pluralityof images which are captured by using a rolling shutter method, andcalculating a position function of a position change of a photographingapparatus, which captures the plurality of images, by using theplurality of captured images.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

These and/or other features and utilities of the present generalinventive concept will become apparent and more readily appreciated fromthe following description of the embodiments, taken in conjunction withthe accompanying drawings of which:

FIG. 1 is a block diagram illustrating a photographing apparatusaccording to an exemplary embodiment of the present general inventiveconcept;

FIG. 2 is a block diagram illustrating a motion estimating apparatusaccording to an exemplary embodiment of the present general inventiveconcept;

FIGS. 3 and 4 are graphs illustrating an operation of the motionestimating apparatus of FIG. 2;

FIGS. 5 through 8D are views illustrating experimental results of aphotographing apparatus according to an exemplary embodiment of thepresent general inventive concept;

FIG. 9 is a view illustrating a motion of a photographing apparatusaccording to an exemplary embodiment of the present general inventiveconcept;

FIG. 10 is a flowchart illustrating an image compensating methodaccording to an exemplary embodiment of the present general inventiveconcept; and

FIG. 11 is a flowchart illustrating a method of calculating a motionaccording to an exemplary embodiment of the present general inventiveconcept.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Reference will now be made in detail to the embodiments of the presentgeneral inventive concept, examples of which are illustrated in theaccompanying drawings, wherein like reference numerals refer to the likeelements throughout. The embodiments are described below in order toexplain the present general inventive concept while referring to thefigures.

FIG. 1 is a block diagram illustrating a photographing apparatus 100 asan electronic apparatus according to an exemplary embodiment of thepresent general inventive concept.

Referring to FIG. 1, the photographing apparatus 100 includes acommunication interface unit 110, a user interface unit 120, a storageunit 130, an image sensing unit 140, a motion estimator 200, an imageprocessor 150, and a controller 160. The photographing apparatus 100 maybe a digital camera, a camcorder, a mobile phone, a portable multimediaplayer (PMP), a webcam, or the like which can continuously capture aplurality of images from one or more objects.

The communication interface unit 110 is formed to connect thephotographing apparatus 100 to at least one terminal apparatus (notillustrated). The communication unit 100 may be accessed in a wirelessor wired way through a local area network (LAN) or the Internet orthrough a universal serial bus (USB) port or a Bluetooth module.

The communication interface unit 110 transmits a signal (or data)corresponding to contents, which are stored in the photographingapparatus 100, to an external terminal apparatus (not illustrated). Indetail, the communication interface unit 110 may transmit a signal (ordata) corresponding to an image or a moving picture file stored in thestorage unit 130, which will be described later, to the externalterminal apparatus or an external server.

If the user interface unit 120 includes a plurality of functional keyswhich are used by a user to set or select various functions supported bythe photographing apparatus 100, the user interface unit 120 may displayvarious types of information. The user interface unit 120 may berealized as a device which simultaneously realizes an input and anoutput like a touch pad. The user interface unit 120 may be realized asa combination of input units, such as a plurality of buttons, and adisplay apparatus, such as a liquid crystal display (LCD) monitor, anorganic light-emitting diode (OLED) monitor, or the like.

The user interface unit 120 receives various control commands, includinga capture start command, a capture end command, etc., from the user. Theuser interface unit 120 also receives setups related to capturing. Indetail, the user interface unit 120 may receive setups including a fileformat type of a captured image, a resolution of the captured image, aframe rate when capturing a moving picture, digital zoom-in/zoom-out, anauto white balance (AWB), an auto focus (AF), an auto exposure (AE),etc.

The user interface unit 120 may be a display unit to display thecaptured image. In detail, if the photographing apparatus 100 iscapturing a moving picture, the user interface unit 120 may display amoving picture captured by the image sensing unit 140 which will bedescribed later. Also, the user interface unit 120 may display varioustypes of contents stored in the storage unit 130 according to a playbackcommand of the user.

The storage unit 130 stores the captured image. In detail, the storageunit 130 may divide an image captured by the image sensing unit 140 intoframes and temporarily store the frames. The storage unit 130 may storean image which is image-processed by the image processor 150, which willbe described later. The storage unit 130 may also store contents (i.e.,a plurality of compressed captured images) which are generated by theimage processor 150 using the image.

The storage unit 130 may be realized as a storage medium of thephotographing apparatus 100 or an external storage medium. For example,the storage unit 130 may be realized as a removable disk, such as a USBmemory, a flash memory, or the like, a storage medium connected to thephotographing apparatus 100, a web server through a network, or thelike.

The image sensing unit 140 continuously captures a plurality of imagesby using a rolling shutter method. In detail, the image sensing unit 140may include a lens, an image sensor, and an analog-to-digital converter(ADC). The lens condenses light of a subject to form an optical image ina photographing area. The image sensor converts the light incidentthrough the lens into an electric signal. The ADC converts an analogsignal into a digital signal and outputs the digital signal as theimage.

The image sensor may include pixels, each formed with a complementarymetal oxide semiconductor (CMOS) optical sensor. Each of the pixels ofthe imaging sensor may read the optical image according to the rollingshutter method. Here, the rolling shutter method refers to a method ofsequentially reading the optical image using one or several pixels of aphotographing area thereof. Hereinafter, an optical image is read in aunit of a pixel line. However, the present inventive concept may beapplied to a case where an optical image is read in a unit of acombination of a plurality of pixel lines or a case where an opticalimage is read in a unit of at least one pixel.

The motion estimator 200 estimates a uniformly accelerated motion of thephotographing apparatus 100 by using a plurality of captured images.Detailed structure and operation of the motion estimator 200 will bedescribed later with reference to FIGS. 2 and 3. As illustrated in FIG.2, the motion estimator 200 may be realized as an additional motionestimating apparatus which separates from the photographing apparatus100 and may be connectable to the photographing apparatus 100 through awired or wireless communication.

The image processor 150 compensates the uniformly accelerated motion ofthe photographing apparatus 100 by using the plurality of capturedimages. In detail, the image processor 150 may compensate the uniformlyaccelerated motion of the photographing apparatus 100 with respect toeach of images captured by the image sensing unit 140, based on auniformly accelerated motion estimated by the motion estimator 200.

In more detail, the image processor 150 calculates a displacement ofeach pixel line on an x axis according to a uniformly accelerated motionwhich is estimated on the x axis by the motion estimator 200. The imageprocessor 150 compensates for a displacement of each pixel line on the xaxis by a displacement of a corresponding pixel line which is calculatedon the x axis. The image processor 150 calculates a displacement of eachpixel line on a y axis according to a uniformly accelerated motion whichis estimated on the y axis by the motion estimator 200. The imageprocessor 150 compensates for a displacement of each pixel line on the yaxis by a displacement of a corresponding pixel line which is calculatedon the y axis in order to compensate for a motion of the photographingapparatus 100 with respect to each of images captured by the imagesensing unit 140.

The image processor 150 calculates displacements of pixel lines on the xand y axes according to uniformly accelerated motions which areestimated by the motion estimator 200 in yaw, pitch, and rolldirections. The image processor 150 compensates for a displacement ofeach pixel line on the y axis by a displacement of a corresponding pixelline, which is calculated on the y axis, to compensate for the motion ofthe photographing apparatus 100 with respect to each of the imagescaptured by the image sensing unit 140.

Also, the image processor 150 may perform signal processing, such asdigital zooming, an AWB, an AF, an AE, or the like, in order to converta format of an image signal output from the image sensing unit 140 or animage signal with a motion compensation of the photographing apparatus100 and to adjust a scale of the image signal.

The image processor 150 may transmit an image captured by the imagesensing unit 140 or a signal-processed image to the user interface unit120 to display the captured image on a display unit.

The image processor 150 may encode the signal-processed image and storethe encoded image in the storage unit 130 so that the captured image isstored. In the present exemplary embodiment, contents of receiving avoice signal have not been described. However, the image processor 150may process the voice signal when the captured image is processed. Thatis, the image processor 150 may combine the captured image with a voicesignal received through an additional internal microphone or an externalmicrophone to generate a moving picture file and store the generatedmoving picture file in the storage unit 130.

The controller 160 may control elements of the photographing apparatus100. In detail, if the image sensing unit 140 continuously captures aplurality of images, the controller 160 controls the motion estimator200 to estimate a uniformly accelerated motion of the photographingapparatus 100. Also, the controller 160 may control the image processor150 to compensate the plurality of captured images based on theestimated uniformly accelerated motion.

As described above, the photographing apparatus 100 according to thepresent exemplary embodiment may estimate a uniformly accelerated motionthereof by using continuously captured images and compensate thecaptured images according to the estimated uniformly accelerated motion.Therefore, the photographing apparatus 100 may compensate for an imagedistortion caused by a rolling shutter thereof. Since the photographingapparatus 100 estimates the uniformly accelerated motion thereof, themotion of the photographing apparatus 100 may be further accuratelyestimated.

Although descriptions of the present exemplary embodiment are based on aCMOS image sensor, the present exemplary embodiment is not limitedthereto. The present general inventive concept may be applied to anothertype of an image sensor which is operable according to a rolling shuttermethod.

FIG. 2 is a block diagram illustrating a motion estimating apparatus 200according to an exemplary embodiment.

Referring to FIG. 2, the motion estimating apparatus 200 may be similarto or the same as the motion estimator 200 of FIG. 1. In detail, themotion estimating apparatus 200 includes an input unit 210, a buffer220, and a calculator 230.

The input unit 210 receives a plurality of images which are captured byusing a rolling shutter method. In detail, the input unit 210sequentially receives a plurality of images captured by the imagesensing unit 140 of FIG. 1. The input unit 210 transmits currentlyreceived images to the buffer 220 and the calculator 230.

The buffer 220 temporarily stores the plurality of images input throughthe input unit 210. The buffer 220 transmits the temporarily storedimages to the calculator 230. In the present exemplary embodiment, theinput unit 210 and the buffer 220 provide a plurality of images to thecalculator 230. However, the buffer 220 may transmit all of theplurality of images to the calculator 230.

The calculator 230 calculates a position function of a position changeof the photographing apparatus 100 which has captured the plurality ofimages by using the plurality of input images. The calculator 230 mayinclude a comparator 240, a velocity calculator 250, and a positioncalculator 260.

The comparator 240 compares the plurality of input images with oneanother to check displacements among the plurality of images. In detail,the comparator 240 compares pixel lines of a plurality of images whichare continuously captured to check a position of a pixel line in anotherimage based on a pixel line of one of the plurality of images in orderto check a displacement of the pixel line. The comparator 240 maycompare preset pixel groups of pixel lines of each of a plurality ofcontinuously captured images to check positions of pixel groups inanother image based on pixel groups of one of the plurality of images inorder to check displacements of the pixel groups. The comparator 240 maycheck displacements of all of a plurality of pixel lines which havecomparable objects.

The comparator 240 may average the displacements of the plurality ofpixel lines to calculate displacements of images.

The velocity calculator 250 calculates a uniform acceleration functionof a motion velocity of the photographing apparatus 100 according to thechecked displacements. In detail, the velocity calculator 250 maycalculate the uniform acceleration function of the motion velocity ofthe photographing apparatus 100 based on the displacements of the pixellines checked by the comparator 240. Since the uniform accelerationfunction requires at least three displacement values, the velocitycalculator 250 may calculate the uniform acceleration function by usingtwo displacement values among three consecutive images. Also, thevelocity calculator 250 may calculate the uniform acceleration functionby using displacement values of a plurality of pixel lines between twoconsecutive images.

If the comparator 240 checks displacements of a plurality of pixellines, the velocity calculator 250 averages the checked displacements ofthe pixel lines to calculate a displacement (or a rotation displacement)of an image (or a frame) and calculates a uniform acceleration functionof a motion velocity (or a rotation velocity) of the photographingapparatus 100 by using the calculated displacement. As described above,the uniform acceleration function requires at least three displacementvalues. In this case, the velocity calculator 250 may calculate theuniform acceleration function by using a displacement (or a rotationdisplacement) among three images. In the above-description, the velocitycalculator 250 calculates a displacement of an image. However, thecomparator 240 may calculate a displacement of an image.

The uniform acceleration function calculated by the velocity calculator250 is a linear function of time as in Equation 1 above. A method ofcalculating constants (an initial velocity and an acceleration constantof a photographing apparatus) of Equation 1 will be described later withreference to FIG. 3.

The position calculator 260 calculates a position function of a positionchange of the photographing apparatus 100 by using the calculateduniform acceleration function. In detail, the position calculator 260may integrate the uniform acceleration function calculated by thevelocity calculator 250 to calculate a position function that is aquadratic function of time. The calculation of the position function maybe performed with respect to each image.

As described above, the motion estimating apparatus 200 according to thepresent exemplary embodiment may estimate a uniformly accelerated motionof the photographing apparatus 100 by using continuously capturedimages. Therefore, the motion estimating apparatus 200 may furtheraccurately estimate a motion of the photographing apparatus 100.

As described with reference to FIG. 2, the calculator 230 calculatesonly a position function with respect to one axis. However, a positionfunction with respect to an x axis and a position function with respectto a y axis are independent of each other. Therefore, the calculator 230may calculate a first position function of the photographing apparatus100 in one axis direction (e.g., on an x axis) by using a method asdescribed above and a second position function in a direction (e.g., ona y axis) perpendicular to the one axis direction by using the samemethod in order to calculate position functions of the photographingapparatus 100 in two axis directions.

The calculator 230 may compare sizes of subjects in a captured image tocalculate a third position function on a z axis. In this case, themotion estimating apparatus 200 may estimate a 3-dimensional uniformlyaccelerated motion of the photographing apparatus 100.

As described above, the calculator 230 calculates only a positionfunction on a coordinate plane. However, the calculator 230 maycalculate a first rotation function in a yaw direction based on a centerof the image sensing unit 140, a second rotation function in a pitchdirection based on the center of the image sensing unit 140, and a thirdrotation function in a roll direction based on the center of the imagesensing unit 140 to calculate rotation functions with respect torotation directions of the photographing apparatus 100.

In the present exemplary embodiment, one uniform acceleration functionis calculated with respect to one axis. However, a captured image may bedivided into a plurality of areas, and uniform acceleration functions ofthe plurality of areas (a plurality of uniform acceleration functions)may be calculated. For example, an image may be divided into upper andlower areas, and uniformly accelerated motions of the upper and lowerareas may be respectively calculated.

As described with reference to FIGS. 1 and 2, only a uniformlyaccelerated motion of the photographing apparatus 100 is estimated.However, if the photographing apparatus 100 is fixed by an apparatussuch as a tripod, and captures a moving subject, not a uniformlyaccelerated motion of the photographing apparatus 100, but an estimationand compensation for a uniformly accelerated motion of the movingsubject may be obtained.

FIGS. 3 and 4 are views illustrating an operation of the motionestimating apparatus 200 of FIG. 2.

A rolling shutter phenomenon occurs when a subject moves or a camerashakes. Therefore, a motion of the photographing apparatus 100 and amotion of the subject are to be considered in order to compensate forthe rolling shutter phenomenon. However, for descriptive convenience, itwill be described that the subject is fixed. In detail, thephotographing apparatus 100 has a frame rate of 30 fps or more.Therefore, it is assumed that a motion of a subject is not greatlydisplaced compared to a motion of the photographing apparatus 100 at theframe rate of 30 fps.

A displacement of the photographing apparatus 100 may be calculated byintegrating a velocity of the photographing apparatus 100. Therefore, amethod of calculating the velocity of the photographing apparatus 100will be described first. A method of calculating a uniform accelerationfunction of a vertical motion of the photographing apparatus 100 willnow be described with reference to FIG. 3.

FIG. 3 illustrates a motion path of the photographing apparatus 100 anda motion path of a pixel line if the photographing apparatus 100vertically moves. Referring to FIG. 3, an x axis denotes time t. Indetail, a capturing completing time of a first image is normalized as 1.A y axis denotes displacements of an image sensor and the photographingapparatus 100. White circles respectively denote pixel lines of theimage sensor, and black circles denote pixel lines which are being read.

If the photographing apparatus 100 moves in a direction of the y axiswhen capturing, reading times of pixel lines are different from oneanother. Therefore, a subject goes down a little more at a reading timeof a second pixel than at a reading time of a first pixel line.

If the photographing apparatus 100 moves at a uniform acceleration, amotion velocity of the photographing apparatus 100 at an n^(th) imagemay be represented as in Equation 1 below:v _(n)(t)=v _(n,0) +a(t−n) n≦t<n+1  (1)wherein t denotes time, n denotes a number of an image, Vn(t) denotes avelocity of the photographing apparatus 100 at the time t, v_(n,0)denotes a velocity of the photographing apparatus 100 when the n^(th)image starts to be captured, and a denotes an acceleration constant.

Equation 1 above may be integrated to be represented as a positionfunction as in Equation 2 below:P(t)={P(n−1)+v _(n−1,0)(t−n+1)+½a _(n−1)(t−n+1)² n−1≦t<nP(n)+v _(n,0)(t−n)+½a _(n)(t−n)² n≦t<n+1  (2)wherein t denotes time, n denotes a number of an image, P(t) denotes aposition of the photographing apparatus 100 at the time t, v_(n,0)denotes the velocity of the photographing apparatus 100 when the n^(th)image starts to be captured, and v_(n−1,0) denotes the velocity of thephotographing apparatus 100 when an n−1^(th) image starts to becaptured. As shown above, the position of the photographing apparatus100 may be represented as a quadratic function of the time t.

A position of a k^(th) pixel line in the n^(th) image may be representedas in Equation 3 below. In detail, the position of the k^(th) pixel linein the n^(th) image is equal to an addition of a position of a k^(th)pixel line in the photographing apparatus 100 to the position P(t) ofthe photographing apparatus 100. In the present exemplary embodiment, itis assumed that the position of the photographing apparatus 100 is fixedto a first pixel line. However, the position of the photographingapparatus 100 may be fixed to a last pixel line or a central pixel line.For the description convenience, the position of the photographingapparatus 100 corresponds to the first pixel line.

$\begin{matrix}{{P_{n}(k)} = {{P( {n + \frac{k}{H}} )} + k}} & (3)\end{matrix}$wherein n denotes a number of an image, k denotes a number of a pixelline, P_(n)(k) denotes the position of the k^(th) pixel line in then^(th) image, and H denotes the total number of pixel lines.

If a subject is fixed, and the photographing apparatus 100 moves in avertical direction, a shape of a particular subject exists on the k^(th)pixel line in the n−1^(th) image. However, after a predetermined timeelapses, the shape of the particular subject may exist on a first pixelline in the n^(th) image. This may be arranged as in Equation 4 below:

$\begin{matrix}{{P_{n - 1}(k)} = {{P_{n}( k_{0} )} = {{{P( {n - 1 + \frac{k}{H}} )} + k} = {{P( {n + \frac{k_{0}}{H}} )} + k_{0}}}}} & (4)\end{matrix}$wherein P_(n−1)(k) denotes the position of the k^(th) pixel line in then−1^(th) image, and P_(n)(k₀) denotes a position of the first pixel linein the n^(th) image.

A position movement of the photographing apparatus 100 may berepresented as in Equation 5 by using Equation 2:P(n)−P(n−1)=v _(n−1,0)+½a _(n−1)  (5)wherein P(n) denotes the position of the photographing apparatus 100 inthe n^(th) image, P(n−1) denotes the position of the photographingapparatus 100 in the n−1^(th) image, V_(n−1,0) denotes an initialvelocity of the photographing apparatus 100 in the n−1^(th) image, anda_(n−1) denotes an acceleration constant of the photographing apparatus100 in the n−1^(th) image.

If the photographing apparatus 100 continuously captures a plurality ofimages, an initial velocity of the photographing apparatus 100 in aparticular image is equal to a last velocity of the photographingapparatus 100 in a previous image. Therefore, the initial velocity maybe represented as in Equation 6 below:v _(n,0) =v _(n−1,0) +a _(n−1)  (6)wherein V_(n,0) denotes the initial velocity of the photographingapparatus 100 in the n^(th) image, v_(n−1,0) denotes the initialvelocity in the n−1^(th) image, and a_(n−1) denotes an accelerationconstant in the n−1^(th) image.

Therefore, if Equations 2 through 6 are arranged, a displacement (or amotion vector) of the k^(th) pixel line in the n−1^(th) image may berepresented as in Equation 7 below:

$\begin{matrix}\begin{matrix}{{{mv}_{n - 1}(k)} = {k - k_{0}}} \\{= {{v_{{n - 1},0}( {1 - \frac{k}{H} + \frac{k_{0}}{H}} )} + {\frac{1}{2}a_{n - 1}}}} \\{( {1 - ( \frac{k}{H} )^{2} + \frac{2k_{0}}{H}} ) +} \\{\frac{1}{2}{a_{n}( \frac{k_{0}}{H} )}^{2}}\end{matrix} & (7)\end{matrix}$wherein mv_(n−1)(k) denotes the motion displacement of the k^(th) pixelline in the n−1^(th) image.

If the motion of the photographing apparatus 100 is not excessive, adisplacement of a pixel line may not be greater than a vertical lengthof the image sensor. This assumption may be represented as in Equation 8below:

$\begin{matrix} {{{mv}_{n - 1}(k)}{\operatorname{<<}H}}arrow{\frac{{mv}_{n - 1}(k)}{H}{\operatorname{<<}1}}  & (8)\end{matrix}$

Therefore, if Equation 8 is applied,

$( {1 - \frac{k}{H} + \frac{k_{0}}{H}} )\mspace{14mu}{and}\mspace{14mu}( {1 - ( \frac{k}{H} )^{2} + \frac{2k_{0}}{H}} )$of Equation 7 may be respectively represented as in Equations 9 and 10:

$\begin{matrix}{\mspace{76mu}{{1 - \frac{k}{H} + \frac{k_{0}}{H}} = {{1 - \frac{{mv}_{n - 1}(k)}{H}} \approx 1}}} & (9) \\{{1 - ( \frac{k}{H} )^{2} + \frac{2k_{0}}{H}} = {{1 - ( \frac{k}{H} )^{2} + \frac{2( {k - {{mv}_{n - 1}(k)}} )}{H}} \approx {1 - ( \frac{k}{H} )^{2} + \frac{2k}{H}}}} & (10)\end{matrix}$

If Equations 9 and 10 are applied to Equation 7, the displacement (orthe motion vector) of the k^(th) pixel line in the n−1^(th) image may berepresented as in Equation 11:

$\begin{matrix}{{{mv}_{n - 1}(k)} = {v_{{n - 1},0} + {\frac{1}{2}{a_{n - 1}( {1 - ( \frac{k}{H} )^{2} + \frac{2k}{H}} )}} + {\frac{1}{2}{a_{n}( \frac{k_{0}}{H} )}^{2}}}} & (11)\end{matrix}$

A calculation of a displacement (hereinafter referred to as a globalmotion vector) of an image by using a displacement of a pixel lineobtained as described above may be performed by using various methods.In detail, a global motion vector may be calculated by using only adisplacement of an arbitrary pixel line or by using an average ofdisplacements of all of pixel lines which include an area matching witha next image. A method of calculating a global motion vector by using anaverage of displacements of all of pixel lines including an areamatching with a next image in order to enhance accuracy will now bedescribed.

If a global motion vector is an average of displacements of all pixellines as described above, the global motion vector may be represented asin Equation 12:

$\begin{matrix}{{G\; M\;{V( {n - 1} )}} = {{\frac{1}{H}{\int_{0}^{H}{{{mv}_{n - 1}(k)}{\mathbb{d}k}}}} = {v_{{n - 1},0} + {\frac{5}{6}a_{n - 1}} + {\frac{a_{n}}{2H^{2}}{\int_{0}^{H}{k_{0}^{2}{\mathbb{d}k}}}}}}} & (12)\end{matrix}$wherein GMV(n−1) denotes the global motion vector in the n−1^(th) image,H denotes the total number of pixel lines, and k₀ denotes a variablerelated to k which is not a constant.

If Equation 8 is used,

$\frac{a_{n}}{2H^{2}}{\int_{0}^{H}{k_{0}^{2}{\mathbb{d}k}}}$of Equation 12 may be represented as in Equation 13 below:

$\begin{matrix}{{{\frac{a_{n}}{2}{\int_{0}^{H}{( \frac{k_{0}}{H} )^{2}{\mathbb{d}k}}}} \approx {\frac{a_{n}}{2}\frac{1}{3}}} = {\frac{1}{6}a_{n}}} & (13)\end{matrix}$

If Equation 13 is applied to arrange Equation 12, Equation 14 below maybe obtained:GMV(n−1)=v _(n−1,0)+⅚a _(n−1)+⅙a _(n) =v _(n−1,0) +a _(n−1)+⅙(a _(n) −a_(n−1))=v _(n,0)+⅙(a _(n) −a _(n−1))  (14)

A global motion vector among continuous images may be represented as inEquation 15 below.(15)GMV(n)−GMV(n−1)=a _(n)+⅙(a _(n−1)−2a _(n) +a _(n−1))(GMV(n)−GMV(n−1)−(GMV(n−1)−GMV(n−2))=a _(n) −a _(n−1)+⅙(a _(n+1)−3a_(n)+3a _(n−1) −a _(n−2))  (15)

If Equations 14 and 15 are combined and arranged, Equation 16 below maybe obtained:

$\begin{matrix}{{{G\; M\;{V( {n - 1} )}} - {\frac{1}{6}( {{G\; M\;{V(n)}} - {2G\; M\;{V( {n - 1} )}} + {G\; M\;{V( {n - 2} )}}} )}} = {{v_{n,0} - {\frac{1}{36}( {a_{n + 1} - {3a_{n}} + {3a_{n - 1}} - a_{n - 2}} )}} = {v_{n,0} - {\frac{1}{36}( {d_{n + 1} - {2d_{n}} + d_{n - 1}} )}}}} & (16)\end{matrix}$wherein d_(n) denotes an acceleration difference between continuousimages. If a frame rate of the photographing apparatus 100 isconsidered, an acceleration difference caused by hand-shaking is notlarge. Therefore, d_(n+1)−2d_(n)+d_(n−1) may be 0.

Therefore, Equation 16 may be arranged as in Equation 17 below.v _(n,0)=GMV(n−1)−⅙(GMV(n)−2GMV(n−1)+GMV(n−2))  (17)

A displacement between images may be calculated by the comparator 240which has been described above, and thus the velocity calculator 250 maycalculate an initial velocity of each of the images by using Equation 17above. The velocity calculator 250 may also calculate an acceleration ofeach of the images by using the calculated initial velocity and Equation6 above.

Therefore, the velocity calculator 250 may calculate a uniformacceleration function of a motion velocity of the photographingapparatus 100 on the y axis by using the calculated initial velocity andthe calculated acceleration.

Also, as described above, the position calculator 260 may integrate thecalculated uniform acceleration function to calculate a positionfunction of a position movement of the photographing apparatus 100 onthe y axis.

As described above, the motion estimating apparatus 200 according to thepresent exemplary embodiment may calculate a position function of thephotographing apparatus 100 by using only Equations 17 and 6 and adisplacement of an image. Therefore, the motion estimating apparatus 200may estimate a motion of the photographing apparatus 100 through arelatively simple calculation. Also, the motion estimating apparatus 200may estimate a motion of the photographing apparatus 100 on theassumption that the photographing apparatus 100 moves at a constantvelocity. Therefore, the motion estimating apparatus 200 may furtheraccurately estimate the motion of the photographing apparatus 100.

An initial velocity and an acceleration in each image may be calculatedthrough the above-described process. Therefore, the calculated initialvelocity and an acceleration constant may be applied to calculate aposition movement of the photographing apparatus 100.

In the present exemplary embodiment, a position function of a verticalmotion of the photographing apparatus 100 is calculated. However, aposition function of a horizontal motion of the photographing apparatus100 may be calculated by using the same method as the above-describedmethod. A method of calculating a uniform acceleration function of thehorizontal motion of the photographing apparatus 100 will now bedescribed with reference to FIG. 4.

FIG. 4 illustrates a motion path of the photographing apparatus 100 anda motion path of a pixel if the photographing apparatus 100 moveshorizontally. Referring to FIG. 4, an x axis denotes displacements ofthe image sensor and the photographing apparatus 100, and a y axisdenotes time.

The position function of the horizontal motion of the photographingapparatus 100 may be represented as in Equation 2 above. However, adisplacement of each pixel line with respect to the horizontal motionmay be represented as in Equation 18 below, differently from Equation 7.

$\begin{matrix}\begin{matrix}{{{mv}_{n - 1}(k)} = {{P( {k + 1} )} - {P(k)}}} \\{= {v_{{n - 1},0} + {\frac{1}{2}a_{n - 1}a_{n - 1}\frac{k}{H}} + {\frac{1}{2}( {a_{n} - a_{n - 1}} )( \frac{k}{H} )^{2}}}}\end{matrix} & (18)\end{matrix}$

Equation 18 may be applied to represent a global motion vector of ann−1^(th) image as in Equation 19 below:

$\begin{matrix}{{G\; M\;{V( {n - 1} )}} = {{\frac{1}{H}{\int_{0}^{H}{{{mv}_{n - 1}(k)}{\mathbb{d}k}}}} = {v_{{n - 1},0} + {\frac{5}{6}a_{n - 1}} + {\frac{1}{6}a_{n}}}}} & (19)\end{matrix}$

If Equation 19 is compared with Equation 14, a global motion vector onthe x axis is equal to a global motion vector on the y axis.

Therefore, the velocity calculator 250 may input a displacement betweenimages on the x axis into Equation 17 to calculate an initial velocityon the x axis. Also, the velocity calculator 250 may calculate anacceleration of each of the images on the x axis by using the calculatedinitial velocity on the x axis and Equation 6 above.

Therefore, the velocity calculator 250 may calculate a uniformacceleration function of a motion velocity of the photographingapparatus 100 on the x axis by using the calculated initial velocity andthe acceleration on the x axis.

Also, the position calculator 260 may integrate the uniform accelerationfunction of the motion velocity on the x axis to calculate a positionfunction of a position movement of the photographing apparatus 100 onthe x axis.

In the above descriptions, the position functions on the x and y axesare calculated. However, a size of a subject of a previous image may becompared with a size of a subject of a next image to calculate aposition function of a position movement of the photographing apparatus100 on a z axis in order to estimate a 3-dimensional motion of thephotographing apparatus 100.

In the present exemplary embodiment, only motions of the photographingapparatus 100 on the x, y, and z axes are estimated. However, theabove-described estimation method may be modified to calculate arotation function of a rotation of the photographing apparatus 100 andestimate a rotation motion of the photographing apparatus 100 throughthe rotation function. In detail, as shown in FIG. 9, the photographingapparatus 100 may rotate in yaw, pitch, and roll directions. Therefore,the velocity function on the axis may be changed to calculate a velocityfunction of the photographing apparatus 100 in the yaw direction as inEquation 20 below. Also, a rotation motion of the photographingapparatus 100 in the yaw direction may be estimated by using thevelocity function in the yaw direction. The velocity function on the yaxis may be changed to calculate a velocity function of thephotographing apparatus 100 in the pitch direction as in Equation 21below. Also, a rotation motion of the photographing apparatus 100 in thepitch direction may be estimated by using the velocity function in thepitch direction. The velocity function on the z axis may be changed tocalculate a velocity function of the photographing apparatus 100 in theroll direction. Also, a rotation motion of the photographing apparatus100 in the roll direction may be estimated by using the velocityfunction in the roll direction.V _(Ψn) =V _(Ψn,0) +a(t−n) n≦t<n+1  (20)wherein t denotes time, n denotes a number of an image, V_(ψn)(t)denotes a rotation velocity of the photographing apparatus 100 in a yawrotation direction ψ at the time t, v_(ψn,0) denotes a rotation velocityof the photographing apparatus 100 in the yaw rotation direction ψ whenthe n^(th) image starts to be captured, and a denotes an accelerationconstant in the yaw rotation direction ψ.V _(θn) =V _(θn,0) +a(t−n) n≦t<n+1  (21)wherein t denotes time, n denotes a number of an image, V_(θn)(t)denotes a rotation velocity of the photographing apparatus 100 in apitch rotation direction θ at the time t, v_(θn,0) denotes a rotationvelocity of the photographing apparatus 100 in the pitch rotationdirection θ when the n^(th) image starts to be captured, and a denotesan acceleration constant in the pitch rotation direction θ.V _(φn) =V _(φn,0) +a(t−n) n≦t<n+1  (22)wherein t denotes time, n denotes a number of an image, V_(φn)(t)denotes a rotation velocity of the photographing apparatus 100 in a rollrotation direction φ at the time t, v_(φn,0) denotes a rotation velocityof the photographing apparatus 100 in the roll rotation direction φ whenthe n^(th) image starts to be captured, and a denotes an accelerationconstant in the roll rotation direction φ.

In the above descriptions, only rotation motions of the photographingapparatus 100 in yaw, pitch, and roll directions are calculated.However, position functions of the photographing apparatus 100 on x, y,and z axes may be calculated, and then rotation functions of thephotographing apparatus 100 in the yaw, pitch, and roll directions maybe calculated by using the position functions on the x, y, and z axes.In other words, an equation in rectangular coordinates may be changed toan equation in spherical coordinates to estimate a rotation motion ofthe photographing apparatus 100.

In the above descriptions, position and rotation motions of thephotographing apparatus 100 are uniformly accelerated motions. However,the position and rotation motions of the photographing apparatus may beestimated on the assumption that the photographing apparatus 100 movesat an acceleration.

FIGS. 5A through 7C are views illustrating experimental results of thephotographing apparatus 100 according to an exemplary embodiment.

Referring to FIG. 5A, three images which are obtained by continuouslycapturing a particular pattern are shown. As shown in FIG. 5A, since thephotographing apparatus 100 moves at a constant velocity to the leftside, the pattern is inclined to the right side as it moves down.

FIG. 5B illustrates a result of a compensated image according to theprior art, and FIG. 5C illustrates a result of a compensated imageaccording to an exemplary embodiment of the present inventive concept.If the results of the compensated images of FIGS. 5B and 5C are comparedwith each other, a motion of a conventional photographing apparatus iscompensated on the assumption that the conventional photographingapparatus moves at a constant velocity. Therefore, a distortion of alower part of the pattern is not properly compensated for. However, amotion of a photographing apparatus according to the present inventiveconcept is compensated on the assumption that the photographingapparatus moves at a constant velocity. Therefore, compensation isprovided for a distortion of a lower part of the pattern at which avelocity of the photographing apparatus is most greatly changed.

FIG. 6 is a view illustrating a photographing result of a photographingapparatus in a vibration environment. Here, the vibration environmentrefers to an environment in which the photographing apparatus vibratesat a vibration period of 6 Hz.

As is illustrated in FIG. 6, an image text is not included, and thus, itis difficult to estimate how each pixel line moves. However, asdescribed above, a motion estimating method according to the presentexemplary embodiment is to estimate a uniformly accelerated motion ofthe photographing apparatus based on a displacement (i.e., a globalmotion vector) of an image. Therefore, a rolling shutter effect may beefficiently compensated for even in this environment.

FIGS. 7A through 7C are views illustrating results of a subject capturedby using a photographing apparatus according to the present inventiveconcept. In detail, FIG. 7A illustrates an original image which isobtained by capturing a subject without an additional compensation. FIG.7B illustrates a captured image which is compensated for by using an OIStechnique, and FIG. 7C illustrates a captured image which is compensatedby using a compensating method according to the present inventiveconcept.

Referring to FIGS. 7A and 7B, if a motion of a photographing apparatusexceeds a displacement provided by the OIS technique, there may not beproper compensation for a rolling shutter phenomenon. However, referringto FIG. 7C, although the photographing apparatus is excessivelydisplaced, the compensating method of the present inventive concept mayeffectively compensate for the rolling shutter phenomenon.

FIGS. 8A through 8D are views illustrating an effect of a photographingapparatus according to an exemplary embodiment.

Referring to FIG. 8A, if the photographing apparatus moves to the rightside due to hand-shaking when a linear subject is captured, thephotographing apparatus captures an image as illustrated in FIG. 8B dueto a rolling shutter effect. Hand-shaking of a person is not a uniformmotion, but a uniformly accelerated motion, and thus, a motion of alower end part of a captured image is largest.

FIG. 8C illustrates a result of compensating the image of FIG. 8B byusing a conventional compensating method (i.e., if it is assumed that amotion of a photographing apparatus is a uniform motion). FIG. 8Dillustrates a result of compensating for the image of FIG. 8B by using acompensating method according to the present inventive concept (i.e., ifit is assumed that a motion of the photographing apparatus is auniformly accelerated motion). According to the conventionalcompensating method, the motion of the photographing apparatus iscompensated for by using the assumption that the motion of thephotographing apparatus is the uniform motion. Therefore, thecompensation is inaccurate for the lower end part of the image which isgreatly displaced, as is illustrated in FIG. 8C. However, according tothe compensating method of the present inventive concept, an image iscompensated for by using the assumption that the motion of thephotographing apparatus is the uniformly accelerated motion. Therefore,the compensation is accurate for the lower end part of the image whichis greatly displaced.

In the present exemplary embodiment, there is compensation for only auniformly accelerated motion. However, if an acceleration constant iscalculated as 0, the photographing apparatus estimates a motion thereofas a uniform motion. Therefore, according to the compensating method ofthe present inventive concept, not only a uniformly accelerated motion,but also a uniform motion, may be easily corrected.

FIG. 10 is a flowchart illustrating an image compensating methodaccording to an exemplary embodiment.

Referring to FIG. 10, in operation S1010, a plurality of images arecontinuously captured. In detail, the plurality of images arecontinuously captured by using a rolling shutter method.

A uniformly accelerated motion of a photographing apparatus iscompensated for by the using the plurality of images. In detail, inoperation S1020, the uniformly accelerated motion of the photographingapparatus is estimated by using the plurality of images. In operationS1030, the plurality of images are compensated for based on theestimated uniformly accelerated motion. The detailed operation ofestimating the uniformly accelerated motion has been described withreference to FIGS. 2 and 3, and thus repeated descriptions will beomitted. Also, the method of providing compensation to the image hasbeen described in detail with reference to FIG. 1, and thus repeateddescriptions will be omitted.

In operation S1040, the compensated image is stored.

As described above, according to the image compensating method of thepresent exemplary embodiment, a uniformly accelerated motion of thephotographing apparatus 100 may be estimated by using continuouslycaptured images. Also, a captured image may be compensated according tothe estimated uniformly accelerated motion, and thus compensation for animage distortion caused by a rolling shutter phenomenon may be provided.In addition, since the uniformly accelerated motion of the photographingapparatus 100 is estimated, a motion of the photographing apparatus 100may be further accurately estimated. The image compensating methodillustrated in FIG. 10 may be performed by a photographing apparatushaving a structure as illustrated in FIG. 1 or other photographingapparatuses having other types of structures.

FIG. 11 is a flowchart illustrating a motion estimating method accordingto an exemplary embodiment.

Referring to FIG. 11, in operation S1110, images from a plurality ofinput images are compared with one another to check displacements amongthe plurality of input images. In detail, pixel lines of each of aplurality of images which are continuously captured may be compared withone another to check a position of a pixel line in another image basedon a pixel line of one of the plurality of images in order to check adisplacement of the pixel line.

In operation S1120, a uniform acceleration function of a motion velocityof a photographing apparatus is calculated according to the checkeddisplacements. In detail, the uniform acceleration function of themotion velocity of the photographing apparatus may be calculated basedon a displacement of a pixel line checked in a previous step.

In operation S1130, a position function of a position change of thephotographing apparatus is calculated based on the calculated uniformacceleration function. In detail, a uniform acceleration functioncalculated in a previous step may be integrated to calculate a positionfunction that is a quadratic function.

As described above, according to the motion estimating method of thepresent exemplary embodiment, a uniformly accelerated motion of thephotographing apparatus 100 may be estimated by using continuouslycaptured images. Therefore, a motion of the photographing apparatus 100may be further accurately estimated. The motion estimating method ofFIG. 11 may be performed by a motion estimating apparatus having astructure as described with reference to FIG. 2, or other motionestimating apparatuses having other types of structures.

The above-described motion estimating method may be realized as at leastone execution program for executing the motion estimating method, andthe execution program may be stored on a computer-readable recordingmedium.

Therefore, blocks of the present inventive concept may be executed ascomputer-recordable codes on the computer-readable recording medium. Thecomputer-readable recording medium may be a device which can storagedata read by a computer system.

Although a few embodiments of the present general inventive concept havebeen shown and described, it will be appreciated by those skilled in theart that changes may be made in these embodiments without departing fromthe principles and spirit of the general inventive concept, the scope ofwhich is defined in the appended claims and their equivalents.

What is claimed is:
 1. A photographing apparatus comprising: an imagesensor configured to continuously receive a plurality of images by usinga rolling shutter method; a motion estimator configured to compare atleast one pixel line in each of the plurality of images, correspondingto one another in the plurality of images, to determine displacements ofthe at least one pixel line in the plurality of images, and estimate auniformly accelerated motion of the photographing apparatus based on thedetermined displacements of the at least one pixel line in the pluralityof images; and an image processor configured to generate an image bycompensating for the uniformly accelerated motion of the photographingapparatus estimated by the motion estimator.
 2. The photographingapparatus as claimed in claim 1, wherein the motion estimator compares aplurality of pixel lines in each of the plurality of images,corresponding to one another in the plurality of images, to determinedisplacements of each of the plurality of pixel lines in the pluralityof images to calculate a plurality of uniform accelerated functions forthe uniformly accelerated motion of the photographing apparatus based onthe displacements of each of the plurality of pixel lines in theplurality of images.
 3. The photographing apparatus as claimed in claim1, wherein the motion estimator comprises: a comparator configured tocompare the at least one pixel line in each of the plurality of imagesto determine the displacements of the at least one pixel line; avelocity calculator configured to calculate a uniform accelerationfunction of a motion velocity of the photographing apparatus to estimatethe uniformly accelerated motion, according to the determineddisplacements of the at least one line; and a position calculatorconfigured to calculate a position function of a position change of thephotographing apparatus by using the calculated uniform accelerationfunction.
 4. The photographing apparatus as claimed in claim 1, whereinthe comparator is configured to compare the at least one pixel line ineach of the plurality of images in a direction selected from x, y, z,yaw, pitch and roll directions to determine displacements of the atleast one pixel line.
 5. The photographing apparatus as claimed in claim1, wherein the motion estimator compares a preset pixel group in each ofthe at least one pixel line in each of the plurality of images, thepreset pixel group corresponding to one another in the plurality ofimages, to determine displacements of the preset pixel group in theplurality of images, and estimate the uniformly accelerated motion ofthe photographing apparatus based on the determined displacements of thepreset pixel group.
 6. The photographing apparatus as claimed in claim1, wherein the motion estimator compares a plurality of pixel lines ineach of the plurality of images, corresponding to one another in theplurality of images, to determine displacements of each of the pluralityof pixel lines in the plurality of images, and averages displacements ofthe plurality of pixel lines in each of the plurality of images in theplurality of images to calculate a uniform accelerated function for theuniformly accelerated motion of the photographing apparatus based on theaveraged displacements of each of the plurality of pixel lines in theplurality of images.
 7. A photographing apparatus comprising: an imagesensor configured to continuously receive a plurality of images by usinga rolling shutter method; a motion estimator configured to estimate auniformly accelerated motion of the photographing apparatus by using theplurality of images; and an image processor configured to generate animage by compensating for the estimated uniformly accelerated motion ofthe photographing apparatus, wherein the motion estimator comprises: acomparator configured to compare the plurality of images with oneanother to determine displacements among the plurality of images; avelocity calculator configured to calculate a uniform accelerationfunction of a motion velocity of the photographing apparatus accordingto the checked displacements; and a position calculator configured tocalculate a position function of a position change of the photographingapparatus by using the calculated uniform acceleration function, and,wherein the uniform acceleration function is a linear function of timeas in an Equation below:v _(n)(t)=v _(n,0) +a(t−n) n≦t<n+1, where t denotes time, n denotes anumber of the plurality of images, Vn(t) denotes a velocity of thephotographing apparatus at the time t, v_(n,0) denotes an initialvelocity of the photographing apparatus at an n^(th) image among theplurality of images, and a denotes an acceleration constant.
 8. Thephotographing apparatus as claimed in claim 7, wherein the initialvelocity v_(n,0) of the photographing apparatus is calculated by usingan Equation below:v _(n,0) =GMV(n−1)−⅙(GMV(n)−2GMV(n−1)+GMV(n−2)), where v_(n,0) denotesthe initial velocity of the photographing apparatus at the n^(th) image,and GMV denotes a displacement of an image among the plurality ofimages.
 9. A photographing apparatus comprising: an image sensorconfigured to continuously receive a plurality of images by using arolling shutter method; a motion estimator configured to estimate auniformly accelerated motion of the photographing apparatus by using theplurality of images; and an image processor configured to generate animage by compensating for the estimated uniformly accelerated motion ofthe photographing apparatus, wherein the motion estimator comprises: acomparator configured to compare the plurality of images with oneanother to determine displacements among the plurality of images; avelocity calculator configured to calculate a uniform accelerationfunction of a motion velocity of the photographing apparatus accordingto the checked displacements; and a position calculator configured tocalculate a position function of a position change of the photographingapparatus by using the calculated uniform acceleration function, andwherein the position calculator integrates the calculated uniformacceleration function to calculate the position function of the positionchange of the photographing apparatus, and wherein the position functionis a quadratic function of time.
 10. The photographing apparatus asclaimed in claim 1, wherein the motion estimator estimates a uniformlyaccelerated motion of the image sensor in one axis direction and auniformly accelerated motion of the image sensor in a directionperpendicular to the one axis direction, based on the displacements ofthe at least one pixel line in the plurality of images.
 11. Thephotographing apparatus as claimed in claim 1, wherein the motionestimator estimates at least one of a uniformly accelerated motion in ayaw direction based on a center of the image sensor, a uniformlyaccelerated motion in a pitch direction based on the center of the imagesensor, and a uniformly accelerated motion in a roll direction based onthe center of the image sensor, to estimate the uniformly acceleratedmotion of the photographing apparatus.
 12. A method of generating animage in a photographing apparatus, the method comprising: continuouslycapturing a plurality of images by using a rolling shutter method; andcomparing at least one pixel line in each of the plurality of images,corresponding to one another in the plurality of images, to determinedisplacements of the at least one pixel line in the plurality of images,and estimating a uniformly accelerated motion of the photographingapparatus based on the determined displacements of the at least onepixel line; and generating an image by compensating for the estimateduniformly accelerated motion of the photographing apparatus.
 13. Themethod as claimed in claim 12, wherein the comparing comprises comparinga plurality of pixel lines in each of the plurality of images,corresponding to one another in the plurality of images, to determinedisplacements of each of the plurality of pixel lines in the pluralityof images to calculate a plurality of uniform accelerated functions forthe uniformly accelerated motion of the photographing apparatus based onthe displacements of each of the plurality of pixel lines in theplurality of images.
 14. The method as claimed in claim 12, wherein thecomparing comprises: comparing the at least one pixel line in each ofthe plurality of images to determine the displacements of the at leastone pixel line; calculating a uniform acceleration function of a motionvelocity of the photographing apparatus to estimate the uniformlyaccelerated motion, according to the determined displacements on the atleast one pixel line; and calculating a position function of a positionchange of the photographing apparatus by using the calculated uniformacceleration function.
 15. The method as claimed in claim 12, whereinthe comparing comprises comparing the at least one pixel line in each ofthe plurality of images in a direction selected from x, y, z, yaw, pitchand roll directions to determine displacements of the at least one pixelline.
 16. The method as claimed in claim 12, wherein the comparingcomprises comparing a preset pixel group in each of the at least onepixel line in each of the plurality of images, the preset pixel groupcorresponding to one another in the plurality of images, to determinedisplacements of the preset pixel group in the plurality of images, andestimating the uniformly accelerated motion of the photographingapparatus based on the determined displacements of the preset pixelgroup.
 17. The method as claimed in claim 12, wherein the comparingcomprises comparing a plurality of pixel lines in each of the pluralityof images, corresponding to one another in the plurality of images, todetermine displacements of each of the plurality of pixel lines in theplurality of images, and averaging displacements of the plurality ofpixel lines in each of the plurality of images to calculate a uniformaccelerated function for the uniformly accelerated motion of thephotographing apparatus based on the averaged displacements of each ofthe plurality of pixel lines in the plurality of images.
 18. A method ofgenerating an image in a photographing apparatus, the method comprising:continuously receiving a plurality of images by using a rolling shuttermethod; estimating a uniformly accelerated motion of the photographingapparatus by using the plurality of images; and generating an image bycompensating for the estimated uniformly accelerated motion of thephotographing apparatus, wherein the estimating comprises: comparing theplurality of images with one another to determine displacements amongthe plurality of images; calculating a uniform acceleration function ofa motion velocity of the photographing apparatus according to thechecked displacements; and calculating a position function of a positionchange of the photographing apparatus by using the calculated uniformacceleration function, and, wherein the uniform acceleration function isa linear function of time as in an Equation below:v _(n)(t)=v _(n,0) +a(t−n) n≦t<n+1, where t denotes time, n denotes anumber of the plurality of images, Vn(t) denotes a velocity of thephotographing apparatus at the time t, v_(n,0) denotes an initialvelocity of the photographing apparatus at an n^(th) image among theplurality of images, and a denotes an acceleration constant.
 19. Themethod as claimed in claim 18, wherein the initial velocity v_(n,0) ofthe photographing apparatus is calculated by using an Equation below:v _(n,0) =GMV(n−1)−⅙(GMV(n)−2GMV(n−1)+GMV(n−2)), where v_(n,0) denotesthe initial velocity of the photographing apparatus at the n^(th) image,and GMV denotes a displacement of an image among the plurality ofimages.
 20. The method as claimed in claim 12, wherein the uniformlyaccelerated motion of the photographing apparatus comprises a firstuniform acceleration function calculated in one axis direction and asecond uniform acceleration function calculated in a directionperpendicular to the one axis direction.
 21. The method as claimed inclaim 12, wherein the uniformly accelerated motion of the photographingapparatus comprises at least one of a uniform acceleration functioncalculated in a yaw direction, a uniform acceleration functioncalculated in a pitch direction, and a uniform acceleration functioncalculated in a roll direction.
 22. A method of generating an image in aphotographing apparatus, the method comprising: continuously receiving aplurality of images by using a rolling shutter method; estimating auniformly accelerated motion of the photographing apparatus by using theplurality of images; and generating an image by compensating for theestimated uniformly accelerated motion of the photographing apparatus,wherein the estimating comprises: comparing the plurality of images withone another to determine displacements among the plurality of images;calculating a uniform acceleration function of a motion velocity of thephotographing apparatus according to the checked displacements; andcalculating a position function of a position change of thephotographing apparatus by using the calculated uniform accelerationfunction, wherein the calculated uniform acceleration function isintegrated to calculate the position function of the position change ofthe photographing apparatus, and wherein the position function is aquadratic function of time.
 23. A non-transitory computer-readablerecording medium comprising a program for executing a method ofgenerating an image, the method comprising: continuously capturing aplurality of images by using a rolling shutter method; and comparing atleast one pixel line in each of the plurality of images, correspondingto one another in the plurality of images, to determine displacements ofthe at least one pixel line in the plurality of images, and estimating auniformly accelerated motion of the photographing apparatus based on thedetermined displacements of the at least one pixel line; and generatingan image by compensating for the estimated uniformly accelerated motionof the photographing apparatus.